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(54) Voice response system 

(57) The present invention relates to system for var- 
ying the voice menus and segments presented to the 
user of a voice response system according to the com- 
petence of the user. The response time of a user to voice 
prompts is measured and an average response time is 
determined. It is assumed that the lower the average 
response time, the greater the competence of the user. 
The average response time is used as an index to a ta- 
ble of ranges of response times. Each range has respec- 
tive voice segments associated therewith. The voice 



segments comprise oral instructions or queries for the 
user and vary according to the anticipated competence 
of the user. If the average response time changes such 
that the voice segments indexed are different to the cur- 
rent voice segments then a data base containing infor- 
mation relating to user competence is updated to reflect 
such a change. Accordingly, when the user next inter- 
acts with the voice response system a new set of voice 
segments more appropriate to the user's competence 
with be played. 
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Description 



The present invention relates to a voice response 
system having dynamic voice menus. 

Voice response systems enable users thereof to ac- 
cess information using a conventional telephone. The in- 
teraction between the users and the system comprises 
various voice prompts output by the system and re- 
sponses thereto input, via the telephone keypad, by the 
user. Voice response systems are used by service pro- 
viders, such as banks, to fully or partially automate tele- 
phone call answering or responding to queries. Typically 
a voice response system provides the capability to play 
voice prompts comprising recorded voice segments or 
speech synthesised from text and to receive responses 
thereto. The prompts are generally organised in the form 
of voice menus invoked by state tables. A state table can 
access and play a voice segment or synthesise speech 
from given text. The prompts are usually part of a voice 
application which is designed to, for example, allow a 
customer to query information associated with their var- 
ious banks accounts. 

An example of such a voice response system is the 
IBM CallPath DirectTalk/6000 product as described in 
■IBM CallPath DirectTalk/6000 General Information and 
Planning" and "IBM CallPath DirectTalk/6000 Voice Ap- 
plication Development" (IBM, DirectTalk, Direct- 
Talk/6000 and CallPath are trade marks of International 
Business Machines Corporation). 

The IBM DirectTalk/600 product provides voice mail 
capabilities. Voice mail provides features such as those 
found in a telephone answering machine together with 
the capability to manipulate any stored messages. For 
example, if a subscriber wishes to listen to the messages 
stored, the voice mail will use a voice response system 
to indicate how many messages have been received, at 
what time and, possibly, from whom. The list of messag- 
es are manipulated using various voice menus or 
prompts presented to the subscriber by the voice re- 
sponse system. The voice response system typically 
asks the subscriber whether or not the messages are to 
be stored or forwarded to another subscriber. 

Facsimile mail systems also use voice response 
systems in a similar manner to voice mail systems. Sub- 
scribers of facsimile system can manipulate stored fac- 
similes or have selected documents faxed to a specified 
facsimile number. Again, the voice response system 
present the subscriber with various options or menus 
which are used to manipulated the facsimiles. Actions 
are selected from the voice men us using the DTMF tones 
generated by conventional DTMF telephones. US patent 
number 4, 918, 722 and US patent number 4,974,254 
disclose methods and system for retrieving facsimile 
data using voice response systems. 

As the users of such system may not be familiar with 
the use thereof, it is necessary to ensure that the instruc- 
tions or voice prompts are sufficiently comprehensive to 
allow an novice user to successfully interact with the sys- 



tem. 

However, the more competent users are in using a 
particular voice response system the more they begin to 
anticipate the various voice prompts and it becomes in- 
5 creasingly tedious for them to have to listen to such com- 
prehensive instructions when more succinct instructions 
would suffice. 

Accordingly, the present invention provides a voice 
response system comprising 
10 means for conducting a telephone call with a user, 
means for storing a plurality of sets of voice prompts, 
each voice prompt comprising at least one voice seg- 
ment which is capable of being played to said user, 
means for selecting one from said plurality of sets of 
15 voice prompts for use during the telephone call, 

means for outputting a voice prompt from said selected 
set of voice prompts to said user and for receiving a re- 
sponse thereto from the user, said response indicating 
to the system the user's requirements. 
20 Therefore, when a particular user instigates, for ex- 
ample, a telephone call the system selects a set from the 
plurality of sets of voice prompts appropriate to the com- 
petence of the user. The sets of voice prompts are varied 
according to the anticipated or actual user competence. 
The use of more succinct voice prompts increases the 
speed with which a competent user can interact with the 
system. 

An embodiment provides a system further compris- 
ing means for determining the competence with which 
30 said caller interacts with said system, and wherein said 
means for selecting selects a voice prompt according to 
the determined competence of the caller. 

Therefore, the competence with which a user inter- 
acts with the system can be measured and the voice 
35 prompts selected for use during the current interaction 
or future interactions can be varied according to said 
measurements. 

A further embodiment provides a system wherein 
the means for determining the competence comprises 
^0 means for determining the response time between play- 
ing a voice prompt to the user and receiving said re- 
sponse thereto, and 

wherein said means for selecting is responsive to said 
means for determining a response time to select a voice 
45 menu according to the response time. 

Therefore, one way in which the competence of the 
user can be gauged is, for example, by measuring the 
response time of a user to a voice prompt and assuming 
that a short response time is indicative of great familiarity 
50 with the voice response system. It is reasonable to as- 
sume that the faster the response the greater the com- 
petence of the user. 

A still further embodiment provides a system where- 
in said means for receiving a telephone call comprises 
55 a plurality of telephone interfaces each capable of receiv- 
ing a telephone call from said caller, and said system fur- 
ther comprises 

means for identifying upon which telephone interface a 
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telephone call made by said caller was received, and 
wherein 

said means for selecting selects a set of voice prompts 
according to which telephone interface received the tel- 
ephone call from said caller. 

If several users have the same telephone, each user 
can be given a telephone number with which they can 
access the system and the voice prompt output to the 
user depends upon which number they used to access 
the system. Each such given number has associated 
therewith respective voice data, said voice data reflect- 
ing the anticipated competence of the users. 

Alternatively, the selection of the set of voice 
prompts can be matched to individual callers. The users 
can be identified in many different ways. For example, if 
different user's have unique respective telephone num- 
bers, the call identification code of a telephone call can 
be used as anMndex to data stored in a user data base 
comprising information relating to the competence of a 
user. 

Alternatively, the user can be asked to enter a pass- 
word before further access is allowed to the system. The 
password can then serve as an index to the stored data 
associated with the user The stored data identifies which 
set of voice data is appropriate for use during an inter- 
action with said user. 

Alternatively, determining the number of times per 
day which a user accesses the system or the length of 
time which a user has subscribed to such a system may 
also be indicative of their competence. 

A further embodiment provides means for determin- 
ing how many times a user has accessed said system, 
and 

means for accessing and amending said stored data ac- 
cording to said determination. 

An embodiment of the present invention will now be 
described, by way of example only, with reference to the 
accompanying drawings in which: 

figure 1 shows schematically a voice response sys- 
tem, 

figure 2 illustrates schematically data structures 
used in a voice response system, namely a voice 
application, voice menu and voice segment tables, 

figure 3 shows schematically the voice application 
of figure 2, 

figure 4 illustrates schematically the voice menu of 
figure 2 comprising voice states for playing and 
receiving responses to voice prompts and determin- 
ing the speed of the response thereto. 

figure 5 shows the voice segment table comprising 
a plurality of voice segments, 

figure 6 illustrates a table comprising a plurality of 



4 

average response times together with respective 
voice segments, 

figure 7 shows a flow diagram illustrating the oper- 
5 ation of the voice menu in the voice response sys- 
tem. 

Referring to figure 1 there is schematically shown a 
voice response system VRS, implemented using Direct- 
10 Talk/6000, comprising an telephony interface Tl, such as 
T1D4-Mode 3 interface for Tl or a CCITT G .703 inter- 
face for El, for receiving from or transmitting to a tele- 
phone T via a communication network CN voice and sig- 
nalling data. The voice response system VRS further 
15 comprises a voice menu data base VMDB in which voice 
menus Voice.menUi to Voice_menu n are stored in the 
form of voice tables together with a plurality of sets of 
voice prompts, a processor for executing voice applica- 
tions, controlling access to the voice menu data base 
20 and, in conjunction with a timer, determining, via a voice 
menu, the average speed of response of a user to the 
voice prompts. The timer is provided via AIX system 
services. Each set of voice prompts comprises voice 
data which are played to the user during an interaction 
25 with the system. Each set of voice prompts is also de- 
signed for different levels of user competence. The sys- 
tem also comprises a user data base UDB which con- 
tains information from which a decision relating to the 
level of competence of the users can be made. The in- 
30 formation allows appropriate voice prompts to be select- 
ed, and used in a voice menu, for user interaction with 
the system. The user data base also contains an identi- 
fication of the current voice segments which are or will 
be used for any such interaction. A voice prompt for a 
35 reasonably competent user may be as follows: 
For local maps and street plans, press 1, 
For country maps and atlases, press 2, 
For other countries, press 3, 
For other types of maps, press 4 
40 It can be seen from the above example that the voice 
prompt comprises all four options in a single voice seg- 
ment. An alternative way of viewing the above voice 
prompts is to consider it as a voice menu comprising a 
number of options to which only one response is re- 
45 quired. Theabove example can also be constructed from 
a plurality of separate voice segments which are played 
contiguously to the caller. 

By contrast a set of voice prompts which may be suit- 
able for a less competent user may be as follows: 
56 For local maps and street plans, press 1 , for next 

item press 3 

(await response) 

For country maps and atlases, press 1 , to go back 
press 2, for next item press 3, 
55 (await response) 

For other countries 1 , to go back press 2, for next 
item press 3, 

(await response) 
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For other types of map, press 1 , to go back press 
2, for next item press 3. 
(obtain response) 
It can be seen in the above example that the voice 
prompt comprises a plurality of voice segments and the 
user has an opportunity to respond to each segment be- 
fore proceeding to the next segment An alternative way 
of viewing the above voice prompt is to consider it to be 
a voice menu comprising a plurality of voice segments 
each of which requires a response from the caller. 

Referring to figure 2, there are schematically shown 
data structures used in the voice response system to fa- 
cilitate interaction with the user. The voice application 
controls the overall commands of the voice response 
system. The voice menu presents to a user at least one 
voice prompt comprising a set of voice segments and 
determines the speed of response thereto. The voice 
menu also updates the stored data concerning the level 
of the user's competence when appropriate. 

Figure 3 shows schematically the voice application. 
Voice applications handle information requests and per- 
form the data processing within a voice response system 
VRS. A voice application typically comprises at least one 
state table having a sequence of states or instructions 
which accomplish the aim of the voice application. Each 
state of the voice application performs a given function 
such as update account or obtain a command from a call- 
er so that a decision as to how to continue can be made. 
Whenever a command is required by the voice applica- 
tion it calls a voice menu. The voice menu outputs to the 
user appropriate voice prompts or segments and re- 
ceives responses thereto. 

Figure 4 illustrates schematically the voice menu. 
The voice menu comprises at least the following states 
which are executed in the order presented: "Reset Tim- 
er" which resets the timer of the voice response system, 
"Play Prompt" which plays at least one identified voice 
segment or set of voice segments, "Receive Input" which 
receives the response to a prompt or voice segment on 
an interrupt driven basis, "Stop Timer" which stops the 
timer and records the time indicated, "Calculate" calcu- 
lates the user's average response time to the voice seg- 
ments (described in further detail below), "Update" up- 
dates, when necessary, the information relating to the 
user's competence which is stored in the user data base, 
"Determine Command" determines which one of a plu- 
rality of commands should be returned to the voice ap- 
plication, and "Return Command" which returns said one 
of a plurality of commands to the voice application. The 
voice menu also has a look-up table which contains the 
plurality of commands and corresponding keys. When 
the user selects a particular key in response to a prompt, 
that key is used to index the corresponding command in 
the table and the command so indexed is returned to the 
voice application. 

The execution of the voice application is determined 
according to the commands returned by the voice men- 
us. The commands are determined by the responses to 



the voice prompts presented to the user. Each applica- 
tion having instigated the playing of a voice prompt ex- 
pects to execute one of a plurality of possible future ac- 
tions. For example, if the following voice menu was 
5 played to the user: 

To exit the system, press 1 ; 
To continue with the query, press 2; 
To return to the previous menu, press 3. 
The voice application will expect to receive from the 
70 voice menu one of three possible commands. The com- 
mands are represented symbolically as, for example, 
EXIT, CONTINUE and RETURN. It can be seen that the 
voice application only receives information relating to 
any future command to be taken and is not concerned 
1& with how a returned command was obtained. For exam- 
ple, if the voice prompt was changed to: 
To exit the system, press 7; 
To continue with the query, press 8; 
To return to the previous menu, press 9. 
20 The application will still receive one of the three sym- 
bolic commands EXIT CONTINUE or RETURN even 
though different keys were depressed to indicate to the 
voice menu the desired command or course of action. 
The advantage of having the main voice application 
25 guided by symbolic commands becomes evident when 
one modifies either voice application or the voice 
prompts. Modifications can be effected to either the voice 
application or voice menu independently of the other. 
Therefore, as illustrated above, a change to the keys 
30 which must be depressed in order to guide the voice ap- 
plication does not necessitate a corresponding modifica- 
tion to the voice application to accommodate such a 
change. The voice menu is configured to expect different 
key depression but maps the different keys to the sym- 
35 bolic commands accordingly In this way the operation 
and maintenance of the main voice application is isolated 
from the user interface or voice menus. 

The sets of voice segments comprise a collection of 
voice segments which can be used to form a voice menu 
40 and to interact with the user. A schematic example of a 
voice segment table is shown in figure 5. For example, 
segment 1 , plays the welcome message heard by users 
when they initially access the system. 

Referring again to figure 1, the timer T is used to 
45 record the elapsed time between the beginning of a voice 
segment and the response thereto. Immediately prior to 
the voice menu playing, for example, voice segments 
embodying a list of options for which a response is re- 
quired, the state, "Reset Timer", which resets and in- 
s o vokes the timer is executed. The user responds to the 
voice segments by pushing one of the keys of their tele- 
phone pad. The system detects the response, on an in- 
terrupt driven basis, by listening for an appropriate DTMF 
frequency in the conventional manner. When a response 
55 to the voice segments is detected the timer is stopped 
and the elapsed time between the beginning of the voice 
segments and the response thereto is determined. 

The voice response system operates as follows. 
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Upon initialisation the voice response system awaits an 
incoming telephone call. The voice response system 
uses the calling identification to retrieve information from 
the user data base which is suitable for use in the voice 
menus. That is, a set of voice prompts are identified 
which is suitable for the forth coming interaction. Accord- 
ingly, the name of the suitable set of voice segments is 
loaded into the voice menu under the heading of "Voice 
Segment Table" as shown in figure 4. 

When the system answers the incoming call it plays 
a prompt welcoming the user as is conveniional and then 
presents the user with a number of options. As the voice 
application at this stage requires an command in order 
tocontinue processing, a voice menu requesting an input 
from the user is invoked. The voice menu resets the timer 
T to zero and then commences playing the voice seg- 
ments 1 to n identified by the "Voice Segment Table" field 
of the voice menu. The user will be expected to respond 
to the voice segments by pressing one of the keys on 
their telephone key pad. When the user presses a key, 
the voice segments currently being played are interrupt- 
ed and the key so pressed is noted by the "Receive Input" 
state. Alternatively, if all of the voice segments have fin- 
ished the system awaits a response for a predetermined 
time. After the predetermined time has elapsed the sys- 
tem assumes that the user has hung up and returns to 
a state awaiting the next incoming call. Assuming the 
user has responded, the timer is stopped via state "Stop 
Timer" and the time is recorded. The call identification of 
the incoming call is used, by the state "Calculation", to 
locate and retrieve information relating to the particular 
user. The information is stored in the user data base 
UDB. The information comprises the following: calling 
number identification number, average response time 
and number of determined response times. The "Calcu- 
lation" state updates the average response time informa- 
tion as follows: 

new average response time = (average response time X 
number of recorded times + current response time)/ 
(number of determined response times + 1). 

The information relating to the number of deter- 
mined response times is also incremented by one and 
stored. The "Update" state updates the stored data in the 
user data base in a manner described below. The "De- 
termine Command" state maps the users response to 
one of the plurality of possible commands expected by 
the voice application. The "Return Command" state re- 
turns the determined command to the voice application. 

Having received a command the voice application 
continues processing in the conventional manner until 
another command is required via another corresponding 
input from the user. 

A suitable state table implementing the above is 
schematically shown in figure 4. 

Once a new average response time has been deter- 
mined a check is made by the "Update" state to ascertain 
whether or not the current set of voice segments is ap- 
propriate to the level of competence of the user. Refer- 



ring to figure 6, there is shown a table comprising ranges 
of response times and respective sets of voice seg- 
ments. The range into which the new average response 
time falls is determined and the corresponding set of 
5 voice segments identified. If the identified voice seg- 
ments are different to current voice segments then the 
voice menu is updated to contain a reference to the iden- 
tified set of voice segments. For example, if the current 
voice segments are voice_segs_1 and the new average 
10 response time fell within the range 10-20, the "Voice 
Segment Table" field of figure 4 would be changed from 
"Voice_segs_1 " to "Voice_segs_2". The data stored in 
the user data base would also be updated to reflect the 
change in voice segment. Accordingly, when the user 
is next accesses the system or when the voice menu is next 
invoked by the voice application the voice segments, 
voice_segs_2, contained within the newly identified set 
of voice segments will be used. 

The sets of voice segments of figure 6 are tailored 
20 to reflect different levels of user competence. It is as- 
sumed that the lower response times are a reflection of 
greater user competence. The average response times 
shown in figure 6 are in arbitrary units. Voice_segs_1 
may comprise the following voice segments which are 
25 all played contiguously 

For local maps and street plans, press 1 , 
For country maps and atlases, press 2, 
For other countries, press 3, 
For other types of maps, press 4. 
30 It can be seen that when using the above the user 
must remember ail of the options or be very familiar 
therewith as there is no opportunity to listen to each in- 
dividual voice segment again. 

A suitable set of voice segments for a Jess compe- 
ls tent user might be voice_segs_5 in which the voice seg- 
ments are played separately as follow: 

For local maps and street plans, press 1 , for next 
item press 3 

(await response) 
40 For country maps and atlases, press 1 , to go back 

press 2, for next item press 3, 
(await response) 

For other countries 1 , to go back press 2, for next 
item press 3, 
45 (await response) 

For other types of map, press 1 , to go back press 
2, for next item press 3. 
(obtain response) 
It can be seen that voice_segs_5 allows the user 
50 more time to consider each voice segment or prompt 
separately and obviates the burden of remembering all 
of the possible options presented. Voice_segs 2 to 4 
would represent sets of graduated voice segments which 
fall between voice_segs_1 and voice_segs_5. 
55 Figure 7 shows a flow diagram schematically illus- 
trating the operation of the voice menu in the voice re- 
sponse system. As a consequence of the voice applica- 
tion requiring an command in order to be able to proceed 
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the voice menu is invoked. Step 700 resets the timer T. 
Step 705 commences playing the voice segments cur- 
rently identified in by "Voice Segment Table" in the voice 
menu. The system awaits an input or is interrupted with 
such an input from the user via their telephone at step s 
710. At step 715 the timer is stopped and the time re- 
corded. At step 720 the voice menu uses the calling iden- 
tification to identify and retrieve stored data indicative of 
the level of competence of the user from the user data 
base. In an embodiment the retrieval of said information io 
can be insitgated before an incoming telephone call is 
answered thereby playing a welcome message which is 
tailored to the callers competance. The data comprises 
an identification of voice segments appropriate to the lev- 
el of competence of the user. The response time to the 15 
voice segments played to the user is determined step 
720. Step 720 also calculates and updates the average 
response time to the voice segments. Next a determina- 
tion is made as to whether or not the data stored in the 
user data base identifying the set of voice segments ap- 20 
propriate to a user should be updated to reflect the fa- 
miliarity of that user with the voice response system. The 
determination is made by comparing the average re- 
sponse time with a series a ranges of response times at 
step 725. The range into which the average response 2s 
time falls is determined together with the associated or 
new voice segments. If the new voice segments are dif- 
ferent to the current voice segments, the data stored in 
the user data base is amended to store a reference to 
the new voice segments thereby reflecting a change in 30 
competence of the user at steps 730 and 735. If the new 
set of voice segments is the same as the current set of 
voice segments there is no need to amend the data 
stored in the user data base. Having amended the stored 
data or determined that there is no need to amend the 35 
data, the command to be returned to the voice applica- 
tion by the voice menu is determined at step 740 as fol- 
lows. The response to the voice segments is used as 
index to the commands table. The command corre- 
sponding to the index or response is then returned to the *o 
voice application at step 745. Execution of the voice 
menu is terminated at step 750 and execution of the 
voice application recommences in accordance with the 
returned command. 

Although the above embodiment gauges user com- <*s 
petence by measuring the response time to voice 
prompt, the present invention is not limited thereto. User 
competence can equally well be determined by, for ex- 
ample, keeping track of the total number of time which a 
user has accessed the system and varying the voice data so 
according to said number. 

Further, a voice response system can be realised 
comprising many or all of the above techniques. For ex- 
ample, having gained initial access to the voice response 
system via one of a plurality of possible telephone inter- ss 
faces and accordingly retrieved voice prompt information 
associated with the telephone interface, the competence 
of the user can be monitored and the various voice 



prompts presented can be varied accordingly. 

Similarly, the embodiment requiring a pass word to 
be enter before allowing full system access can also in- 
clude the means for measuring the speed of response 
of the user and varying the stored data and voice 
prompts accordingly. 

The present invention can be used to vary either the 
voice menus used by the voice application or the voice 
segments accessed by the voice menus. 

Although embodiments have been described in 
which the voice menus are dynamically varied during a 
call the present invention is not limited thereto. An em- 
bodiment can equally well be realised in which an aver- 
age response time for a complete interaction with the 
user is determined and the stored data is accessed and 
amended only after the conclusion of the interaction. 
Such an embodiment would reduce the amount of time 
the system needed to update data associated with the 
user. 

Further, even though the above embodiments de- 
scribe systems in terms of an incoming call, the present 
invention can equally well be used to tailor the voice 
prompts for outgoing calls. The system, prior to instigat- 
ing a call to a particular user, would identify from the user 
data base, using, for example, the user's telephone 
number, stored data indicative of the level of competance 
of the user. The identified stored data would then be used 
to determine suitable sets of voice prompts to be used 
for the interaction with the caller user. 



Claims 

1 . A voice response system comprising 
means for conducting a telephone call with a user, 
means for storing a plurality of sets of voice prompts, 
each voice prompt comprising at least one voice 
segment which is capable of being played to said 
user, 

means for selecting one from said plurality of sets of 
voice prompts for use during the telephone call, 
means for outputting a voice prompt from said 
selected set of voice prompts to said user and for 
receiving a response thereto from the user, said 
response indicating to the system the user's require- 
ments. 

2. A system as claimed in claim 1 , further comprising 
means for determining the competence with which 
said user interacts with said system, and wherein 
said means for selecting selects a voice prompt 
according to the determined competence of the 
user. 

3. A system as claimed in claim 2, wherein said means 
for determining the competence comprises 
means for determining the response time between 
playing a voice prompt and receiving said response 
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thereto, and 

wherein said means for selecting is responsive to 
said means for determining a response time to 
select a set of voice prompts according to the 
response time. 5 

4. A system as claimed in either of claims 2 or 3, further 
comprising 

means, responsive to said means for determining 
the competence with which said user interacts with io 
said system, for accessing and amending stored 
data reflecting said competence. 



5. A system as claimed in claim 1 , wherein said means 

for receiving a telephone call comprises a plurality *5 
of telephone interfaces each capable of receiving a 
telephone call from said user, and said system fur- 
ther comprises 

means for identifying upon which telephone inter- 
face a telephone call made by said user was 20 
received, and wherein 

said means for selecting selects a set of voice 
prompts according to which telephone interface 
received the telephone call from said user. 

25 

6. A system as claimed in any preceding claim, further 
comprising 

means for identifying the user, and 
means, responsive to said identification, for access- 
ing stored data associated with the user, the stored 30 
data being indicative of the competence of the user, 
and wherein 

said means for selecting is responsive to said stored 
data. 

35 

7. A system as claimed in claim 6, further comprising 
means for recording how many times a user has 
interacted with the system, and 

means for accessing and amending said stored data 
according to said determination. 40 



8. A system as claimed in any preceding claim, further 
comprising voice menus and means for modifying 
said sets of voice segments independently of said 
voice menus, said means for modifying comprising 
memory for storing a table comprising a plurality of 
indices, each index corresponding to a possible user 
response, and respective data corresponding to 
respective commands, wherein said voice menus 
are responsive to said commands, 
means for indexing using a user response one of 
said plurality of commands, and 
means for guiding the operation of said system 
according to said one of said plurality of commands. 
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